Papers with multimodal neural machine translation
Multimodal Neural Machine Translation Using Synthetic Images Transformed by Latent Diffusion Model (2023.acl-srw)
Copied to clipboard
| Challenge: | Existing methods to translate source language sentences using images are not optimal for machine translation. |
| Approach: | They propose a new multimodal neural machine translation model using synthetic images transformed by a latent diffusion model. |
| Outcome: | The proposed model improves translation performance on English-German translation tasks using the Multi30k dataset. |
Supervised Visual Attention for Multimodal Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies show that a conventional visual attention mechanism trained in an unsupervised manner is not effective for multimodal neural machine translation. |
| Approach: | They propose a supervised visual attention mechanism for multimodal neural machine translation that captures the relationship between a word and an image region more precisely than a conventional visual attention system. |
| Outcome: | The proposed model improves on English-German and German-English translation tasks and English-Japanese and Japanese-English tasks using the Flickr30k Entities JP dataset. |